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8  July  2008 

1.  SCOPE. 


a.  This  document  describes  the  procedures  for  determining  the  mobility  capabilities  and 
characteristics  of  remotely  and  teleoperationally  controlled  Unmanned  Ground  Vehicle  (UGVs). 
It  focuses  on  the  contribution  of  teleoperation  to  overall  mobility  and  safe  operation. 

b.  These  procedures  are  intended  to  build  upon  the  mobility  related  Test  Operating 
Procedures  (TOPs)  called  out  in  TOP  2-2-540 1  and  to  set  the  stage  for  testing  autonomous 
mobility  capabilities  and  characteristics.  Tests  of  lethal  and  non-lethal  weaponized  UGVs  must 
refer  to  TOP  2-2-5422. 


2.  FACILITIES  AND  INSTRUMENTATION. 

2.1  Facilities. 

Item 

Enclosed  facility 
Variable  attenuators 
Areas  with  intersecting  roads  and 
narrow  alleys 

2.2  Instrumentation- 


Devices  for  Measuring 

Permissible  Measurement  Uncertainty 

Radio  Frequency  Resolution 

±  15  kHz 

Bandwidth 

RF  Transmitted  And  Received 

±  .1%  of  value 

Power 

RF  Attenuation 

±  ldB 

OCU  Position 

±  0.5  m 

Vehicle  Position  And  OCU 

±  0.5  m 

Vehicle  Speed 

±  .4  km/hr 

Vehicle  Heading 

±1° 

Pitch  And  Roll  Rate 

±  100°/sec 

Vehicle  Computer  Temperature 

±  1°C 

Vehicle  Data  Link  Transceiver 

±  1°C 

Temperature 

Vehicle  Power  Bus  Voltage 

±  1  Volt 

Time 

±  0.1  sec 

Independent  Video  Of  Vehicle 

30  frames  per  second,  Infrared  (IR)  or 

Movement 

color  (based  on  test  conditions). 

Acceleration 

±  5%  of  reading  or  .  lg  (whichever  is 
greater). 

Superscript  numbers/letters  correspond  to  those  in  Appendix  C,  References 

Requirement 
Vehicle  lab  testing 
±.5  dB 
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2.2.1  The  selection  of  instrumentation  and  data  collection  equipment  installed  for  UGV  mobility 
testing  should  consider  the  following: 

a.  Equipment  Mass  Properties.  The  size  and  weight  of  instrumentation  and  data 
recorders  must  be  in  proportion  to  the  size  and  weight  of  the  test  article.  The  use  of 
disproportionately  large  or  heavy  equipment  may  have  an  unacceptable  effect  on  test  results. 
Where  feasible,  weight  should  be  removed  to  keep  mass  properties  (center  of  gravity,  weight, 
etc.)  constant  when  an  instrumentation  package  is  added. 

b.  Real-Time  Data  Downloading.  Wireless  transmission  of  test  data  may  interfere  with 
nonnal  system  operation  due  to  electromagnetic  interference.  Prior  to  test  execution, 
compatibility  between  real-time  data  downlinks  and  system  operation  must  be  verified. 

c.  Spectrum  Analyzers.  This  equipment  is  needed  to  identify  the  presence  of  other 
frequencies  in  use  at  any  given  time  in  the  test,  and  should  be  operated  in  the  vicinity  of  the 
UGV  and  the  Operator  Control  Unit  (OCU).  Although  systems  are  required  to  have  a  frequency 
authorization  prior  to  the  start  of  testing,  establishing  cause  and  effect  during  UGV  mobility 
testing  requires  a  means  of  ensuring  that  frequency  interference  is  not  a  factor  in  test  results. 

d.  UGV  System  Data  Tap.  A  significant  source  of  data  in  a  test  effort  relative  to  UGV 
mobility  will  come  from  UGV  system  data  taps,  especially  for  advanced  autonomous 
capabilities.  This  infonnation  is  useful  for  establishing  cause  and  effect  when  examining  test 
results,  as  well  as  for  monitoring  the  test  article  to  ensure  safety  on  the  test  range.  System  data 
taps  and  supporting  data  harvesting  and  analysis  tools  must  be  compliant  with  the  data  bus 
standards  of  the  test  article.  Many  emerging  UGV  systems  are  compliant  with  the  Joint 
Architecture  for  Unmanned  Systems  (JAUS)  standard.  The  Joint  Ground  Robotics  Enterprise 
(JGRE),  formerly  known  as  the  Joint  Robotics  Program,  established  JAUS  as  a  standard 
architecture  for  UGVs  developed  within  the  JGRE. 

3.  REQUIRED  TEST  CONDITIONS. 

a.  UGV  and  independent  emergency  shutdown  system  operators  must  meet  training 
requirements  for  test  article  operation,  test  area  familiarization,  and  local  SOPs. 

b.  Software  testing  must  be  sufficiently  completed  as  defined  in  TOP  2-2-540. 

c.  The  test  article  must  comply  with  the  safety  and  instrumentation  requirements  defined 
in  TOP  2-2-540,  including  the  following: 

(1)  For  any  UGV  capable  of  injuring  or  killing  a  person,  an  independent  emergency 
shutdown  system  must  be  properly  installed  and  its  functionality  must  be  verified  prior  to  any 
other  testing  or  training  activities. 

(2)  UGVs  must  be  instrumented  to  support  the  critical  infonnation  analysis  and 
displays  as  needed  by  the  test  officer  to  support  test  control  and  monitoring. 
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(3)  Unique  warning  lights  that  warn  nearby  personnel  of  vehicle  movement  must  be 
installed.  It  must  be  verified  that  operation  of  these  lights  does  not  interfere  in  any  way 
(physically  or  electronically)  with  the  system  under  test  (SUT). 

d.  Standoff  distances  from  the  UGV  must  be  established  for  personnel  and  equipment  in 

forward,  lateral,  and  rearward  directions.  Standoffs  must  consider  where  the  vehicle  could  go 
during  uncommanded  movement  and  full  throttle  acceleration. 

4.  TEST  PROCEDURES. 

4. 1  Vehicle  Subsystem  Tests.  These  tests  identify  key  operating  characteristics  associated  with 
UGV  subsystems.  Completing  these  tests  first  in  a  UGV  test  program  are  part  of  an  overall 
strategy  to  slowly  build  confidence  in  UGV  system  robustness  and  to  prepare  the  tester  for 
additional  phases  of  testing. 

4.1.1  Tethered  Controller  Test.  This  test  identifies  the  operating  characteristics  ofUGVs  while 
under  control  of  a  hand-held  controller  that  plugs  directly  into  the  vehicle.  Such  a  device  is 
typically  used  for  vehicle  movement  within  maintenance  facilities,  for  loading  or  unloading  the 
vehicle  from  a  trailer,  or  anytime  vehicle  movement  on  a  test  range  cannot  be  perfonned  via  RF 
data  link.  It  also  tests  for  lockouts  that  prevent  other  controllers  from  taking  control  of  the 
vehicle. 

a.  Ensure  the  OCU  is  powered  off. 

b.  Perform  the  Vehicle  Fine  Control  subtest  (paragraph  4.4.3)  using  the  tethered 
controller. 


c.  Power  up  the  OCU  and  look  for  indications  that  the  tethered  controller  has  control  of 
the  vehicle. 

d.  Use  the  OCU  to  attempt  to  take  control  of  the  vehicle.  Record  indications  of 
controlling  the  unit  before  and  after  the  OCU  control  attempt. 

e.  Disconnect  the  tethered  controller  and  allow  OCU  to  take  control  of  the  vehicle. 

f.  Plug  tethered  controller  back  into  the  vehicle  and  attempt  to  control  vehicle.  Record 
observations  that  indicate  the  tethered  controller  now  controls  the  vehicle. 

4. 1 .2  System  Field  of  View  Test.  This  test  identifies  regions  around  the  vehicle  that  cannot  be 
seen  by  the  remote  operator  via  teleoperation,  and  identifies  the  effort  required  to  detect  people 
or  impairments  to  mobility  near  the  vehicle. 

a.  Use  the  procedures  outlined  in  TOP  3-2-812  to  obtain  a  field  of  vision  of  sensors  as 
presented  through  the  OCU.  Be  sure  to  address  each  extreme  in  sensor  pan,  tilt,  and  field  of 
view  (FOV)  setting.  Measure  the  FOV  of  each  sensor  at  each  range  setting. 
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b.  Record  the  list  of  operator  actions  and  timelines  required  to  use  one  sensor  to  observe 
a  blind  spot  of  another  sensor. 

c.  Using  collected  data  identify  residual  blind  spots.  Residual  blind  spots  are  areas  that 
cannot  be  observed  by  switching  between  views  or  by  combining  views. 

d.  Determine  the  size  and  location  of  residual  blind  spots.  Address  the  likelihood  that 
people  or  objects  large  enough  to  impair  mobility  could  fit  within  residual  blind  spots. 

4.1.3  Vehicle  State  Reporting  Test.  This  test  is  used  to  identify  variations  between  the  reported 
and  actual  vehicle  states.  Accurate  state  reporting  is  essential  for  ensuring  that  the  vehicle  is  safe 
enough  to  approach  and  handle  and  for  perfonning  certain  tasks  according  to  the  intended 
mission. 

a.  Identify  key  vehicle  data  reported  to  the  OCU.  States  needed  for  safe  operation  during 
mission  execution  include  position,  speed,  heading,  steering,  pitch  and  roll  attitude,  and  reporting 
sensor  viewing  angles.  States  needed  to  ensure  that  the  vehicle  is  safe  to  approach  include  gear 
and  brake  settings,  engine  power  level,  vehicle  electrical  power  state,  and  the  states  of  any 
articulating  parts  (e.g.:  turrets,  arms,  etc.). 

b.  Record  both  the  OCU  reported  vehicle  data  and  the  vehicle  truth  data.  Obtain  truth 
data  from  test  instrumentation  independent  of  the  system  and  from  visual  observation  of  the 
system. 


c.  Identify  variations  between  reported  and  actual  states  that  may  affect  safe  vehicle 
operation. 

4.2  Emergency  Shutdown  Test.  This  test  provides  confidence  to  the  testers  that  the  emergency 
shutdown  system  can  be  relied  upon  to  shut  the  UGV  down  in  a  manner  that  enables  it  to  be 
safely  approached.  This  subtest  is  intended  for  both  RF  and  panic  stop  button  shutdown  systems. 
Of  particular  importance  is  the  testing  of  independent  emergency  shutdown  systems  that  may  be 
applied  to  some  systems.  Conduct  in  conjunction  with  the  Data  Link  Degradation  Effects  subtest 
(paragraph  4.3)  with  the  following  additions: 

a.  Identify  each  available  means  of  shutting  down  a  vehicle  in  an  emergency  on  a  test 
range  should  the  vehicle  become  uncontrollable.  These  may  include  test  center  applied 
independent  kill  systems,  prime  item  developer  independent  kill  systems,  or  any  other  system 
that  is  not  intended  to  be  a  part  of  the  UGV  system  and  that  does  not  use  the  system’s  data  link. 

b.  Restrain  or  confine  the  vehicle,  then  power  it  up  and  establish  a  slow  wide  turn. 

Wheels  should  be  pivoted  slightly  and  turning. 

c.  Use  an  emergency  shutdown  system  to  shut  down  the  vehicle.  Record  the  results  of 
each  shutdown. 

d.  Repeat  for  each  means  of  shutting  down  the  system  in  an  emergency. 
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4.3  Data  Link  Degradation  Effects.  This  test  identifies  the  effects  of  data  link  degradation  on 
overall  system  performance  and  operating  characteristics. 

a.  In  a  laboratory  setting,  hardwire  the  OCU  and  the  vehicle  together  with  variable 
attenuators  between  them  with  a  low  initial  attenuation  setting. 

b.  Using  mission  planning  software  on  the  system,  enter  the  time  period  the  UGV  is  to 
wait  before  executing  lost  link  procedures.  If  available,  select  a  lost  link  behavior  with 
discernable  changes  in  vehicle  function  (e.g.:  hard  turn,  back  up,  etc.). 

c.  Operate  the  vehicle  in  a  restrained  or  confined  area  within  the  facility  according  to  a 
defined  set  of  planned  functions.  Include  maneuvers  that  challenge  obstacle  negotiation  and 
collision  avoidance  with  degraded  and/or  delayed  imagery. 

d.  Incrementally  increase  the  attenuation  by  a  consistent  amount  and  repeat  vehicle 
operation. 


e.  As  affects  become  apparent  record  the  attenuation  setting  and  observations  on  the 
effects  of  data  link  degradation  on  both  vehicle  operation  and  OCU  imagery.  Record  evidence  of 
degradation  effects. 

f.  Continue  until  data  link  is  lost. 

g.  Record  observations  on  UGV  actions  and  OCU  indications  when  data  link  is  lost. 

h.  Allow  vehicle  to  remain  in  lost  link  state  for  3  minutes  or  20%  longer  than  what  was 
programmed  for  lost  link  procedures,  whichever  is  longer.  This  allows  time  for  any 
undocumented  vehicle  behaviors  to  become  apparent. 

i.  Decrease  attenuation  by  half  the  amount  by  which  it  was  being  increased  and  wait  30 
seconds  for  data  link  reacquisition  to  occur.  Repeat  this  step  until  link  is  reestablished. 

j.  Record  observations  on  UGV  actions  and  OCU  indications  when  data  link  is  restored. 

4.4  Automotive  Performance.  These  tests  identify  overall  system  performance  under  each 
mode  of  control. 

4.4.1  Tailored  Automotive  Tests.  Conduct  automotive  tests  as  defined  in  TOP  2-2-540.  For 
each  of  the  following  subtests  include  the  following  information  as  appropriate  to  intended 
mission: 

a.  Grades  and  Slopes.  Record  the  information  made  available  to  the  remote  operator 
regarding  slope  angle  and  impending  vehicle  roll  over.  Record  remote  operator  indications  of 
nose  slide  or  tail  slide.  Conduct  for  each  mode  of  control. 
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b.  Standard  Obstacles.  For  each  mode  of  control,  maneuver  the  vehicle  into  position  and 
conduct  one  trial  of  negotiating  the  standard  obstacles.  Record  observations  on  OCU  control 
inputs  required  for  steering  effort.  Record  issues  associated  with  maneuvering  the  vehicle  up  to 
or  over  the  obstacles. 

c.  Fording.  Identify  effects  of  water  near  or  on  driving  sensors  to  include  degradation  of 
imagery  at  the  OCU.  Record  indications  of  loss  of  controllability  or  traction  available  to  the 
remote  operator.  Record  difficulties  with  water  entry  or  exit  associated  with  each  mode  of 
control. 

d.  Soft  Soil  Mobility.  Record  indications  of  loss  of  controllability  or  traction  available  to 
the  remote  operator. 

e.  Steering  and  Handling.  For  each  mode  of  control,  record  steering  and  heading 
indications  available  to  the  remote  operator.  Record  observations  on  suitability  of  the 
information  available  to  the  remote  operator  (steering  and  handling  indications  and  the  OCU 
fields  of  view)  to  support  steering  and  handling  related  functions.  Record  observations  on  OCU 
control  inputs  required  for  steering  effort. 

f.  Acceleration  /Max  Speed.  Conduct  for  each  mode  of  control.  Record  observations  of 
OCU  indications  of  vehicle  speed  and  acceleration. 

g.  Braking.  Conduct  for  each  mode  of  control.  Record  observations  of  OCU  indications 
of  vehicle  speed  and  braking  effort. 

4.4.2.  Test  for  Driver  Assist  Functions.  Driver  assist  functions  relieve  the  operator  of  certain 
burdens  in  operating  the  vehicle  remotely  or  offer  a  degree  of  vehicle  protection  against  operator 
actions  that  could  inadvertently  impair  vehicle  motion.  They  DO  NOT  relieve  the  remote 
operator  of  the  fundamental  task  of  driving  the  vehicle.  Examples  include  speed  control,  heading 
control,  and  autonomous  rollover  avoidance  to  correct  operator  inputs  that  would  otherwise 
cause  a  rollover. 

a.  During  automotive  tests  described  above,  verify  that  known  driver  assist  functions  can 
be  activated  and  deactivated  as  needed  and  that  they  function  as  intended. 

b.  Record  OCU  indications  of  the  state  of  driver  assist  functions. 

c.  Record  observations  on  system  operation  with  driver  assist  function  on  and  off. 

d.  Over  the  course  of  the  overall  test  program  record  additional  observations  that  may 
imply  the  presence  of  undocumented  driver  assist  capabilities. 

4.4.3  Vehicle  Fine  Control.  Although  much  of  the  conventional  automotive  testing  examines 
the  limits  of  automotive  performance,  this  test  is  intended  to  examine  the  fine  control  inputs  and 
responses  needed  for  some  situations.  This  test  may  be  combined  with  the  Tailored  Automotive 
Tests  (paragraph  4.4.1). 
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a.  Acceleration. 

(1)  For  each  mode  of  control,  instruct  the  remote  operator  to  drive  at  set  speeds  using 
smooth  transitions  between  each  speed. 

(2)  Record  observations  on  the  system’s  ability  to  finely  adjust  acceleration  and 

speed. 

b.  Braking. 

(1)  For  each  mode  of  control,  instruct  the  remote  operator  to  drive  at  set  speeds  using 
smooth  transitions  between  each  speed.  Record  observations  on  the  system’s  ability  to  finely 
reduce  vehicle  speed  through  braking. 

(2)  Identify  a  point  in  the  distance  and  instruct  the  remote  operator  to  stop  the  vehicle 
just  in  front  of  that  spot.  Record  observations  on  the  ability  of  the  vehicle  to  be  brought  to  a 
gentle  stop  at  a  designated  point. 

c.  Steering. 

(1)  For  each  mode  of  control,  instruct  the  remote  operator  to  move  the  vehicle 
through  a  figure  eight  course  with  defined  edges  no  greater  than  50%  of  vehicle  width.  Course 
must  include  at  least  one  straight  section.  One  turn  should  require  a  turn  radius  equal  to  the 
minimum  demonstrated  turn  radius  plus  25%. 

(2)  Operate  the  vehicle  at  approximately  10%  of  maximum  demonstrated  speed  for 

two  laps. 

(3)  Record  observations  on  the  ability  of  the  vehicle  to  be  kept  within  the  lane. 
Record  observations  on  the  system’s  steering  resolution. 

4.5  Teleoperation  Effectiveness.  This  test  examines  how  well  the  overall  operating 
characteristics  of  the  UGV  work  together  to  support  the  soldier  in  employing  a  teleoperated 
UGV. 

4.5.1  OCU  Usefulness.  While  many  teleoperated  systems  have  similar  components  and 
displays,  huge  variations  may  exist  in  the  ease  with  which  the  UGV  can  be  operated.  This  test 
examines  the  role  of  the  OCU  in  enabling  the  remote  operator  to  effectively  use  the  UGV. 

a.  Observe  remote  operators  and  collect  their  comments  throughout  the  test  effort 
regarding  usefulness  of  available  controls  and  information  to  maneuver  the  vehicle  as  needed. 

b.  Obtain  OCU  video  captures  as  needed  to  support  remote  operator  comments. 
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4.5.2  Corner  Negotiation.  This  test  is  intended  to  identify  UGV  safe  operating  limitations  due  to 
difficulties  the  remote  operator  may  have  in  seeing  the  area  into  which  the  vehicle  is  about  to 
turn  or  in  maneuvering  the  vehicle  into  a  narrow  area. 

a.  Beginning  at  off-road  speed,  slow  the  vehicle  as  needed  to  make  turns  onto 
intersecting  roads.  Two  of  the  intersecting  roads  must  be  narrow:  one  road  must  be 
approximately  75%  wider  than  the  UGV,  and  one  must  be  approximately  50%  wider  than  the 
UGV.  Turns  must  be  made  onto  the  narrow  intersecting  roads  from  both  the  left  and  the  right. 
Turns  must  be  conducted  in  both  daytime  and  in  nighttime  conditions. 

b.  Conduct  at  least  3  trials  of  each  condition. 

c.  Observe  the  remote  operator’s  actions  needed  to  clear  the  area  before  making  the  turn. 
Note  the  timelines  associated  with  switching  OCU  displays  between  looking  ahead  of  the  vehicle 
and  looking  into  the  turn. 

d.  Collect  comments  from  remote  operators  and  independent  observers  on  the  suitability 
of  teleoperation  capabilities  to  safely  turn  the  UGV. 

4.6  Environmental  Operating  Condition  Tests.  These  tests  are  intended  to  identify  operating 
characteristics  of  teleoperated  UGVs  that  result  from  exposure  to  extremes  in  environmental 
conditions. 

a.  Select  environmental  tests  as  defined  in  TOP  2-2-540  based  on  system  requirements 
documents,  intended  missions,  and  mission  locations. 

b.  Conduct  Vehicle  Fine  Control  (paragraph  4.4.3)  and  Teleoperation  Effectiveness 
(paragraph  4.5)  subtests  while  exposed  to  required  environmental  conditions. 

c.  Record  the  environmental  effects  on  overall  system  performance  and  safety.  For  key 
changes  in  system  performance,  specifically  address  the  environmental  effects  that  caused  the 
performance  reduction. 

4.7  System  Anomalies.  This  section  outlines  additional  information  to  be  collected  during  the 
test  program. 

a.  Early  in  the  test  effort,  identify  OCU  indications  of  abnonnal  vehicle  operation,  and 
assess  the  safety  status  of  the  vehicle  during  each  problem. 

b.  During  each  anomaly  encountered  in  the  test  effort  make  note  of  the  remote  operator’s 
ability  to  detennine  that  a  problem  exists  and  to  understand  the  nature  of  the  problem. 

c.  Document  all  problems  that  involve  the  vehicle  being  in  a  state  that  is  unsafe  to 
approach. 
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d.  Identify  general  recommended  improvements  to  system  documentation  based  on 
observed  system  capabilities  and  characteristics. 

5.  DATA  REQUIRED. 

5.1  Vehicle  Subsystem  Tests. 

5.1.1  Tethered  Controller  Test  Data. 

a.  Data  from  Vehicle  Fine  Control  test. 

b.  List  of  indications  at  OCU  that  tethered  controller  has  taken  control  of  vehicle. 

c.  List  of  indications  at  OCU  that  OCU  has  taken  control  from  tethered  controller. 

d.  OCU  can  take  and  retain  control  of  vehicle  during  tethered  control.  (Y/N) 

e.  Tethered  controller  can  take  and  retain  control  while  under  OCU  teleoperation  control. 
(Y/N) 

5.1.2  Vehicle  Blind  Spot  Test  Data. 

a.  Field  of  view  (degrees  horizontal,  degrees  vertical)  for  each  setting  of  each  sensor  at 
each  extreme  in  pan  and  tilt. 

b.  Azimuth  and  elevation  boundaries  of  residual  blind  spots  for  areas  that  cannot  be 
observed  with  any  combination  of  sensors  selectable  at  the  OCU. 

c.  Size  and  locations  of  blind  spots. 

d.  List  of  operator  actions  required  to  observe  single  sensor  blind  spots. 

e.  Time  required  to  use  one  sensor  to  observe  the  blind  spot  of  another  sensor  (sec). 

(1)  Sensor  identification,  pan  and  tilt  setting,  and  FOV  setting  that  has  a  blind  spot. 

(2)  Sensor  identification,  pan  and  tilt  setting,  and  FOV  setting  that  is  used  to  observe 
the  blind  spot. 

f.  Residual  blind  spot  assessment: 

(1)  A  person  can  fit  in  the  residual  blind  spots  while  crouched.  (Y/N) 

(2)  The  residual  blind  spots  can  contain  an  object  large  enough  to  prevent  vehicle 
motion.  (Y/N) 
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(3)  The  remote  operator  can  see  enough  via  the  OCU  to  maneuver  the  vehicle  in 
confined  areas.  (Y/N) 

5.1.3  Vehicle  State  Reporting  Test  Data  (Reported  and  Truth). 

a.  Orientation  and  motion  states: 

(1)  Heading  (deg). 

(2)  Pitch  and  roll  angles  (deg). 

(3)  Speed  (km/hr). 

(4)  Direction  of  travel  (forward,  reverse). 

(5)  Position. 

b.  Actuator  commanded  and  reported  states: 

(1)  Throttle  (%). 

(2)  Braking  (%). 

(3)  Steering  (deg  from  centerline,  direction  of  deflection). 

(4)  Gear  (gear  id). 

(5)  Vehicle  electrical  power. 

(6)  States  of  articulating  parts  (locked,  unlocked,  etc.). 

(7)  OCU  sense  feed  states. 

5.2  Emergency  Shutdown  Test  Data. 

List  of  indicators  of  Emergency  Shutdown  System  degradation. 

5.3  Data  Link  Degradation  Effects. 

a.  Lost  link  mission  plan  settings. 

b.  Response  and  controllability  data. 

(1)  Time  command  was  issued  by  operator  (sec). 

(2)  Time  vehicle  began  to  respond  (sec). 
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c.  Observed  Effects. 

(1)  Screen  captures  or  video  captures  of  OCU  imagery  showing  each  stage  of 
imagery  degradation. 

(2)  List  of  attenuation  settings  and  noted  observations  on  OCU  display  quality  and 
vehicle  controllability. 

(3)  Observations  on  changes  in  system  performance. 

5.4  Automotive  Performance. 

5.4. 1  Tailored  Automotive  Tests.  Supplemental  data  requirements  for  automotive  performance 
TOPs  include  the  following: 

a.  Grades  and  Slopes. 

(1)  Mode  of  control. 

(2)  OCU  indications  of  slope  angle,  pitch  and  roll,  and  impending  roll  over. 

(3)  OCU  indications  of  nose  slide  or  tail  slide. 

(4)  Observations  on  suitability  of  OCU  displays  for  operation  on  slopes  and  grades. 

b.  Standard  Obstacles. 

(1)  Mode  of  control. 

(2)  Time  elapsed  from  first  contact  of  vehicle  with  an  obstacle  to  when  the  obstacle 
was  completely  passed  (sec). 

(3)  List  of  extraordinary  remote  operator  actions  (actions  other  than  those  needed  for 
nonnal  vehicle  operation)  required  to  negotiate  the  obstacles. 

(4)  Observations  of  issues  associated  with  maneuvering  the  vehicle  up  to  or  over 

obstacles. 

c.  Lording. 

(1)  Mode  of  control. 

(2)  List  of  effects  of  water  near  or  on  driving  sensors  to  include  degradation  of 
imagery  at  the  OCU. 

(3)  List  of  OCU  indications  of  loss  of  traction. 
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(4)  Observations  on  difficulties  with  water  entry  or  exit. 

d.  Soft  Soil  Mobility. 

(1)  Mode  of  control. 

(2)  List  of  OCU  indications  of  loss  of  traction. 

e.  Steering  and  Handling. 

(1)  Mode  of  control. 

(2)  List  of  OCU  steering  and  heading  indications. 

(3)  Observations  on  information  available  to  the  remote  operator  (steering  and 
heading  indication  and  OCU  fields  of  view)  to  maneuver  the  vehicle. 

f.  Acceleration  /Max  Speed: 

(1)  Mode  of  control. 

(2)  Commanded  speed. 

(3)  Achieved  steady-state  speed. 

(4)  Observations  on  the  system’s  ability  to  finely  adjust  acceleration  and  speed. 

g.  Braking. 

(1)  Mode  of  control. 

(2)  Commanded  speed. 

(3)  Achieved  steady-state  speed. 

(4)  Observations  on  the  system’s  ability  to  finely  adjust  deceleration  and  speed. 

(5)  Description  of  marking  used  to  identify  the  intended  stopping  location. 

(6)  Observations  on  the  system’s  ability  to  brake  to  a  designated  spot. 

5.4.2  Test  for  Driver  Assist  Capabilities. 

a.  Descriptions  of  observed  driver  assist  capabilities. 

b.  Test  conditions  under  which  driver  assist  functions  were  observed. 
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c.  OCU  indications  of  driver  assist  status. 

d.  Remote  operator  can  engage  and  disengage  driver  assist  functions  as  needed.  (Y/N) 

e.  Driver  assist  is  only  “on”  with  remote  operator’s  knowledge.  (Y/N) 

f.  List  of  observed  driver  assist  capabilities. 

5.4.3  Vehicle  Fine  Control. 

a.  Max  speed:  meters  per  second. 

b.  Speed  Resolution:  Number  of  speed  variations  available,  or  “Continuously  Variable” 
if  a  very  high  number. 

c.  Acceleration  response:  time  to  accelerate  from  zero  to  top  speed  (sec). 

d.  Decelerate  response:  time  to  slow  to  zero  using  only  throttle  control  (sec). 

e.  Brake  resolution:  Able  to  incrementally  adjust  braking  effort.  (Y/N) 

f.  Brake  response:  distance  covered  when  stopping  from  full  speed  to  zero  (m). 

g.  Steering  resolution:  Number  of  steering  variations  available  under  to  one  side,  or 
“Continuously  Variable”  if  a  very  high  number. 

h.  Steering  response  time:  time  required  to  go  from  straight  ahead  to  full  deflection  in 
one  direction,  (sec) 

i.  Observations  on  ability  of  the  remote  operator  to  maintain  specified  courses,  slow 
speeds,  and  to  brake  and  accelerate  gently. 

5.5  Teleoperation  Effectiveness. 

5.5.1  OCU  Usefulness. 

a.  Assessment  of  overall  image  quality,  controls,  and  vehicle  states  to  help  the  remote 
operator  maneuver  the  vehicle  around  obstacles  and  through  tight  areas. 

b.  Description  of  the  situations  in  which  remote  operator  did  not  understand  what  was 
happening  at  the  vehicle. 

c.  Screen  captures  or  photos  highlighting  infonnation  supplied  to  remote  operator  while 
maneuvering  the  vehicle. 
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5.5.2  Corner  Negotiation. 

a.  List  of  actions  required  for  the  remote  operator  to  observe  the  area  into  which  the 
vehicle  is  about  to  turn. 

b.  Observations  on  the  success  rate  of  completing  turns  into  narrow  paths. 

c.  Observations  on  the  ability  of  the  remote  operator  to  look  in  one  direction  using 
vehicle  sensors  while  the  vehicle  is  moving  in  another. 

d.  Observations  on  the  time  and  the  care  the  remote  operator  has  to  put  into  making 

turns. 

5.6  Environmental  Operating  Condition  Test  Data  Requirements. 

a.  Data  as  defined  in  environmental  TOPs  and  standards. 

b.  Data  as  defined  in  Vehicle  Fine  Control  and  Corner  Negotiation  subtests. 

c.  List  of  effects  on  OCU:  menu  responsiveness,  control  freedom  of  movement,  screen 
resolution,  brightness,  contrast,  and  overall  image  quality  for  each  sensor  FOV  and  setting. 

d.  List  of  effects  on  the  vehicle:  vehicle  responsiveness,  vehicle  fine  control,  and 
controllability. 

e.  Observations  on  ability  of  the  OCU  to  be  used  to  maneuver  the  vehicle  safely  and 
effectively. 

f.  Assessment  of  video  display  to  support  full  vehicle  mobility  at  required  speeds  during 
bright  sunlight  and  in  total  darkness. 

5.7  System  Anomalies. 

a.  List  of  uncommanded  or  unresponsive  situations. 

(1)  Occurrences  of  incorrect  status  reporting. 

(2)  Occurrences  of  uncommanded  movement  or  state  changes. 

(3)  Occurrences  of  violating  safe  boundaries  in  time  or  in  space. 

(4)  Occurrences  of  abrupt  system  behaviors. 

b.  Observations  regarding  recurring  anomalies  (actions  leading  up  to  anomaly, 
environments,  etc.). 
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c.  Anomaly  Resolution. 

(1)  Observations  on  sufficiency  of  information  presented  to  the  remote  operator  to 
identify  anomalies. 

(2)  Observations  on  ability  of  the  remote  operator  to  remotely  correct  and  resume 

mission. 

(3)  Data  from  vehicle  data  tap. 

(4)  Data  from  OCU  data  tap. 

d.  List  of  anomalies  causing  unsafe  conditions. 

e.  Training  and  Documentation  Improvements. 

(1)  Remote  operator  comments  on  system  operation  and  mobility. 

(2)  List  of  discrepancies  between  actual  vehicle  operation  and  available 
documentation. 

(3)  List  of  inadvertent  operator  errors  while  maneuvering  and  controlling  the  vehicle. 
6.  PRESENTATION  OF  DATA. 

6.1  Vehicle  Subsystem  Test  Results. 

6.1.1  Tethered  Controller  Test  Results. 

a.  Present  a  table  showing  maximum  automotive  performance  demonstrated  during 
tethered  controller  operation  for  speed,  steering,  and  braking. 

b.  Describe  the  automotive  characteristics  of  the  UGV  under  tethered  control  in  context 
of  vehicle  operation  in  facilities.  Describe  control  responsiveness  and  sensitivity  to  throttle, 
steering,  and  braking  commands  to  enable  both  smooth  changes  for  normal  operation  as  well  as 
abrupt  changes  needed  to  suddenly  avoid  a  collision.  Provide  a  recommendation  for  the  number 
of  people  needed  to  act  as  safety  observers  while  under  tethered  control  to  protect  nearby 
personnel,  facility  equipment,  and  the  test  article. 

c.  Describe  lockouts  that  prevent  OCU  control  during  tethered  control  as  well  as  order  of 
precedence  available  when  a  tethered  controller  is  activated  during  OCU  control. 

d.  Describe  the  suitability  of  the  tethered  controller  for  test  support  activities  such  as 
maneuvering  the  vehicle  into  and  out  of  facilities,  on  and  off  ramps  for  transporting  the  vehicle, 
and  getting  the  vehicle  out  of  impaired  mobility  situations. 
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6.1.2  Vehicle  Blind  Spot  Test  Results. 

a.  Present  side  and  top-down  views  of  the  UGV  showing  coverage  of  each  sensor  aboard 
the  vehicle. 

b.  For  systems  that  enable  combined  views  or  that  enable  the  remote  operator  to  shift 
between  views,  present  one  additional  set  of  views  (side  and  top-down)  showing  sensor  coverage 
areas  superimposed. 

c.  Describe  the  size  of  the  residual  blind  spots  that  cannot  be  observed  by  switching  or 
combining  views  that  could  contain  a  person  (standing  or  crouched). 

d.  Describe  the  operator  actions  needed  to  detect  people  and  impairments  to  mobility 
(e.g.:  posts,  trees)  near  the  vehicle  such  as  changing  sensor  selections  or  settings  during  system 
operation.  Describe  the  consequences  of  those  required  actions  on  safe  operation  of  the  system 
around  people,  equipment,  and  facilities. 

6.1.3  Vehicle  State  Reporting  Test  Results. 

a.  Describe  the  ability  of  the  vehicle  to  promptly  and  accurately  report  its  state  to  the 
OCU  in  the  areas  of:  1)  Vehicle  orientation  and  motion  information  such  as  heading,  pitch  and 
roll  attitude,  speed,  and  direction  of  travel;  2)  commanded  and  reported  automotive  control 
settings  for  throttle,  brake,  steering,  and  gearing;  and  3)  driving  sensor  state  information  such  as 
sensor  selection,  sensor  field  of  view,  sensor  orientation  settings  such  as  pan  and  tilt,  and  sensor 
discrimination  settings  such  as  white-hot/black-hot. 

b.  Describe  the  suitability  of  reported  information  to  safely  operate  the  vehicle. 

c.  Describe  the  suitability  of  reported  information  to  know  when  the  vehicle  is  safe  to 
approach. 

6.2  Emergency  Shutdown  Test  Results. 

a.  Describe  indicators  of  data  link  degradation  on  each  available  emergency  shutdown 
system. 

b.  Describe  the  overall  effectiveness  for  each  available  emergency  shutdown  system. 
Describe  the  effects  of  emergency  shutdown  system  activation  on  the  vehicle  along  with  the 
timeliness  of  each  of  those  effects.  Also,  describe  sufficiency  of  available  indicators  to  the 
remote  operator  and  to  the  independent  shutdown  system  operator  that  the  system  is  safe  to 
approach  and  handle. 

6.3  Data  Link  Degradation  Effects. 

a.  Present  a  table  or  graph  showing  data  link  attenuation  at  each  major  change  in  system 
responsiveness  or  change  in  video  quality  sufficient  to  affect  teleoperation  of  vehicle  movement. 
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b.  Describe  the  observed  effects  of  data  link  degradation  on  system  responsiveness  and 
OCU  imagery.  Address  changes  in  system  response  timeliness  and  the  rate  at  which  this 
degradation  occurs  (rapid,  very  gradually,  etc.).  Describe  how  changes  in  OCU  imagery  affect 
the  ability  of  a  remote  operator  to  maneuver  the  vehicle  safely.  Include  screen  captures  and 
associated  conditions  as  needed  to  help  describe  the  relationship  between  data  link  degradation 
and  overall  controllability  and  mobility  of  the  system. 

6.4  Automotive  Performance. 

6.4.1  Tailored  Automotive  Tests. 

Present  the  results  of  each  automotive  test  as  required  by  each  automotive  TOP  under  each  mode 
of  control  along  with  supplemental  data  defined  in  paragraph  5.4.1. 

6.4.2  Test  for  Driver  Assist  Functions. 

Describe  any  observed  driver  assist  capabilities  such  as  autonomous  rollover  avoidance  actions, 
heading  control,  or  speed  control.  Describe  ability  of  remote  operator  to  engage,  disengage,  and 
observe  current  state  of  any  driver  assist  functions.  Describe  the  observed  functionality  of  each 
driver  assist  function. 

6.4.3  Vehicle  Fine  Control. 

Describe  overall  ability  of  the  system  to  apply  automotive  capabilities  in  each  mode  of  control. 
Include  a  narrative  on  the  ability  of  the  vehicle  to  be  lined  up  properly  to  traverse  obstacles  in  a 
timely  manner,  to  be  stopped  at  intended  points,  and  to  be  turned  at  intended  locations. 

6.5  Teleoperation  Effectiveness. 

6.5.1  OCU  Usefulness. 

Describe  the  usefulness  of  vehicle  controls,  states,  and  imagery  to  support  teleoperation. 

6.5.2  Corner  Negotiation. 

Describe  the  remote  operator’s  actions  needed  to  clear  the  area  into  which  the  vehicle  will  make 
sharp  turns.  Describe  the  extent  to  which  the  operator  can  safely  clear  the  area  into  which  the 
vehicle  will  turn  while  ensuring  the  area  is  clear  ahead  of  the  vehicle.  Describe  safety 
implications  of  demonstrated  comer  negotiation  capabilities  and  limitations. 

6.6  Environmental  Operating  Condition  Testing. 

a.  Describe  the  environmental  effects  on  Vehicle  Fine  Control  and  Teleoperation 
Effectiveness. 
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b.  Describe  the  environmental  effects  on  OCU  controls.  Describe  the  effects  on  display 
responsiveness,  brightness,  contrast,  and  resolution. 

c.  Describe  the  environmental  effects  on  vehicle  responsiveness  and  controllability. 

d.  Describe  the  environmental  effects  on  data  link  performance. 

e.  Summarize  the  overall  environmental  effects  on  system  performance  and  safety. 

6.7  System  Anomalies. 

a.  Present  a  table  summarizing  uncoimnanded  activities  and  unresponsive  situations  with 
associated  conditions. 

b.  Clarify  or  highlight  specific  abnormalities  as  needed  and  cite  TIRS,  video  clips,  or 
other  sources  if  useful. 

c.  Describe  the  sufficiency  of  information  available  to  the  operator  to  remotely  identify 
and  correct  anomalies  and  continue  with  a  task.  Identify  anomalies  that  the  system  can  correct 
on  its  own  (i.e.:  do  not  involve  remote  operator  actions)  and  enable  task  continuation  without 
reprogramming  or  re-initialization. 

d.  List  the  anomalies  that  leave  the  vehicle  in  an  unsafe  state  to  approach  and  require 
subsequent  activation  of  the  emergency  shutdown  system. 

e.  Describe  the  areas  where  training  or  documentation  needs  emphasis  to  ensure  safe 
operation  of  the  system. 

f.  Provide  simple  recommendations  for  system  improvement. 
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APPENDIX  A.  ABBREVIATIONS. 


DoD 

Department  of  Defense 

FOV 

Field  of  View 

IR 

Infrared 

JAUS 

Joint  Architecture  of  Unmanned  Systems 

JGRE 

Joint  Ground  Robotics  Enterprise 

OCU 

Operator  Control  Unit 

RF 

Radio  Frequency 

SUT 

System  under  test 

TIRS 

Test  Incident  Reports 

TOP 

Test  Operations  Procedure 

UGV 

Unmanned  Ground  Vehicle 
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Forward  comments,  recommended  changes,  or  any  pertinent  data  which  may  be  of  use  in 
improving  this  publication  to  the  following  address:  Test  Business  Management  Division 
(TEDT-TMB),  US  Army  Developmental  Test  Command,  314  Longs  Corner  Road 
Aberdeen  Proving  Ground,  MD  21005-5055.  Technical  information  may  be  obtained 
from  the  preparing  activity:  TEDT-AT-AD-F,  U.S.  Army  Aberdeen  Test  Center,  400 
Colleran  Rd.,  APG,  MD  21005-5055.  Additional  copies  are  can  be  requested  through  the 
following  website:  http://itops.dtc.army.mil/RequestForDocuments.aspx,  or  through  the 
Defense  Technical  Information  Center,  8725  John  J.  Kingman  Rd.,  STE  0944,  Fort 
Belvoir,  VA  22060-6218.  This  document  is  identified  by  the  accession  number  (AD 
No.)  printed  on  the  first  page. 


